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Program description 

Lions Quest Skills for Adolescence is a schoolwide program 
designed for middle school students (grades 6-8). The program 
was designed to promote good citizenship skills, core character 
values, and social-emotional skills and discourage the use of 
drugs, alcohol, and violence. The program includes a classroom 
curriculum, schoolwide practices to create a positive school 

climate, parent and family involvement, and community involve- 
ment. The curriculum may vary in scope and intensity, lasting 
from nine weeks to three years. The lessons use cooperative 
group learning exercises and classroom management tech- 
niques to improve classroom climate. A related program is 
reviewed in the intervention report on Skills for Action. 

Research 

One study of Skills for Adolescence met the What Works 
Clearinghouse (WWC) evidence standards. This study included 

more than 7,000 students from 34 middle schools in the Los 
Angeles, Washington, DC-Baltimore, and Detroit areas. 


Effectiveness Skills for Adolescence was found to have potentially positive effects on students’ behavior. 


Behavior 


Knowledge, attitudes, and values Academic achievement 


Rating of effectiveness Potentially positive effects 
Improvement index 1 2 Average: +2 percentile points 

Range: -1 to +11 percentile 
points 


Not reported 
Not reported 


Not reported 
Not reported 


1. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

2. These numbers show the average and the range of improvement indices for all findings across the study. 
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Developer and contact 

Lions Quest, Lions Clubs International Foundation, Professional 
Mailing Distribution Center (PMDS), PO Box 304, Annapolis 
Junction, MD 20701-0304. Email: info@lions-quest.org. Web: 
www.lions-quest.org Telephone: (800) 446-2700. 

Scope of use 

Skills for Adolescence, grades 6-8 is one of a suite of programs 
developed by Lions Quest. Other programs in the set include 
Skills for Growing, grades K-5 and Skills for Action, grades 9-12. 
The program, developed in 1985, is in its fourth revised edition. 
According to the developer, more than 50,000 teachers have been 
trained in Skills for Adolescence in the United States and more 
than 150,000 have been trained in 30 countries worldwide. The 
number of students participating in some parts of Skills for Ado- 
lescence alone has reached more than 1.7 million in the United 
States and about 2.7 million worldwide. Skills for Adolescence 
may have changed since the studies were conducted. The WWC 
recommends asking the developer for information about the most 
current version of this curriculum and taking into account that 
student demographics and school context may affect outcomes. 

Teaching 

Skills for Adolescence consists of 102 lessons in nine units 
based on building such positive character values as respect for 
others, personal and social responsibility, appreciation of diver- 
sity, good citizenship, ethics of service to the community, healthy 
life habits, and social and emotional skills. Year 1 lessons are 
organized into eight units, and an additional service-learning 
unit, taught in conjunction with these units, is infused into the 
curriculum. Year 2 and Year 3 booster units address healthy life 
habits and drug prevention. Lessons can be presented in nine- 
week mini-course, quarter, semester, year-long, or multi-year 
formats or integrated into existing curricula. Teaching methods 
include group work activities, skills practice, and classroom 
or community service projects. The positive school climate 


component is operated through a school climate committee that 
includes teachers, students, parents, and community members. 

The developer provides such teacher resources as curriculum 
materials, a curriculum manual, a teacher’s resource guide, a 
Drugs Information Guide, a parent meeting guide, The Surprising 
Years book for parents, Changes and Challenges workbooks 
for students, and booster sessions for concept reinforcement 
when using a three-year implementation format. Lessons are 
intended to be taught by trained, Lions Quest certified teachers. 
Curriculum materials are only available to trained teachers. 
Training consists of a two- or three-day workshop. According to 
the developer, a large range of targeted staff development in- 
service workshops and an extensive 10-day training-of-trainers 
program, intended to prepare schools to conduct their own staff 
development, are also available. 

Cost 

Training workshops can be organized locally for $4,000 for a 
two-day workshop or $5,000 for a three-day workshop, with 
an added cost of $500 if the reservation is made less than six 
weeks before the training. The developer also offers workshops 
at regional locations for $500 a person (with discounts for four or 
more registrants and four or more weeks advance registration). 
With a guarantee of 12 or more participants, the developer will 
provide this workshop on a by-request basis. Collaboration with 
a local Lion’s Club group can reduce workshop prices. 

Curriculum sets are included in the price of the training. 
Additional sets and upgrades to the newest edition are available 
for purchase by trained individuals. A curriculum set in English 
(4th edition) costs $120; a set in Spanish (3rd edition) costs $99. 
Changes and Challenges student workbooks are available for 
$3.50 a copy for the 3rd edition in English or $5.00 a copy in 
Spanish and $5.95 a copy for the 4th edition in English. The Sur- 
prising Years parent books cost $3.95 a copy for both English 
(4th edition) and Spanish (3rd edition) versions. A set of both 
books is available for $7.95, with volume pricing available. 
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Research 


Effectiveness 


The WWC found Skills 
for Adolescence to have 
potentially positive 
effects on behavior 


WWC Intervention Report 


Nine studies reviewed by the WWC investigated the effects of 
Skills for Adolescence. One study (Eisen, Zellman, & Murray, 
2003) was a randomized controlled trial that met WWC evidence 
standards. The remaining 8 studies did not meet WWC evidence 
screens. The Eisen, Zellman, and Murray (2003) study focused 
on Skills for Adolescence implemented as a schoolwide inter- 
vention. 

Eisen, Zellman, and Murray (2003) investigated program 
effects on students in seventh grade and again on the same 
students in eighth grade. The study included 34 middle schools 


pair-matched and randomly assigned to study conditions from 
within pairs. The schools were located in three large metro- 
politan areas in Los Angeles, California; Detroit, Michigan; 
and the Washington, DC-Baltimore, Maryland area. This study 
compared behavioral outcomes for students participating in 
the Skills for Adolescence program with outcomes for students 
who participated in other drug education and prevention 
practices (ranging from school assemblies to the Drug Abuse 
Resistance Education, or DARE, program) common for these 
grade levels. 


Findings 

The WWC review of character education addresses student 
outcomes in three domains: behavior; knowledge, attitudes, and 
values; and academic achievement. 

Behavior. Eisen, Zellman, and Murray (2003) reported statisti- 
cally significant differences favoring the intervention group on three 
drug-related outcomes (marijuana — lifetime use, marijuana — use 
in the last 30 days, and binge drinking during the last 30 days 
by baseline binge drinkers) about one year after the end of the 
program. The program’s effect on one of these outcomes (binge 
drinking) was confirmed to be statistically significant as calculated 
by the WWC. 3 No statistically significant effects were found for the 
lifetime or recent use of cigarettes and other illicit substances or 


binge drinking during the last 30 days by baseline nonbinge drink- 
ers. The average effect size for the domain was neither statistically 
significant nor substantively important (less than 0.25). 4 

Rating of effectiveness 

The WWC rates interventions as positive, potentially positive, 
mixed, no discernible effects, potentially negative, or negative. 
The rating of effectiveness takes into account four factors: the 
quality of the research design, the statistical significance of the 
findings (as calculated by the WWC 3 ), the size of the difference 
between participants in the intervention condition and the com- 
parison condition, and the consistency in findings across studies 
(see the WWC Intervention Rating Scheme) . 


Improvement index 

For each outcome domain, the WWC computed an improve- 
ment index based on the effect size (see the Technical Details 
of WWC-Conducted Computations) . The improvement index 
represents the difference between the percentile rank of 
the average student in the intervention condition versus the 
percentile rank of the average student in the comparison 
condition. Unlike the rating of effectiveness, the improvement 


index is entirely based on the size of the effect, regardless of 
the statistical significance of the effect, the study design, or the 
analysis. The improvement index can take on values between 
-50 and +50, with positive numbers denoting favorable results. 
The average improvement index for eighth-grade students’ 
behavior is +2 percentile points, with a range of -1 to +11 
percentile points. 


3. The level of statistical significance was calculated by the WWC and, where necessary, corrects for clustering within classrooms or schools and for mul- 
tiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the Technical Details of WWC-Conducted Computations for the formulas 
the WWC used to calculate the statistical significance. In the case of Skills for Adolescence, corrections for multiple comparisons were needed. 

4. Short-term findings assessed for seventh grade are presented in the technical appendices and were not used for rating purposes. 
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The WWC found Skills 
for Adolescence to have 
potentially positive effects 
on behavior (continued) 

References 


Summary 

The WWC reviewed 9 studies on Skills for Adolescence. 5 One of 
these studies met WWC evidence standards. This study found 
potentially positive effects on eighth-grade students’ behavior. 


Character education, an evolving field, is beginning to establish 
a research base. The evidence presented in this report is limited 
and may change as new research emerges. 


Met WWC evidence standards 

Eisen, M., Zellman, G. L., & Murray, D. M. (2003). Evaluating the 
Lions-Quest “Skills for Adolescence” drug education pro- 
gram: Second-year behavior outcomes. Addictive Behaviors, 
28, 883-897. 

Additional sources: 

Eisen, M. (2002). Intermediate outcomes from a life skills edu- 
cation program with a media literacy component. In Crano, 
W. D., & Burgoon, M. (Eds.) Mass media and drug preven- 
tion: Classic and contemporary theories and research, (pp. 
187-214). Mahwah, NJ: Lawrence Erlbaum. 

Eisen, M., Zellman, G. L., Massett, H. A., & Murray, D. M. 
(2002). Evaluating the Lions-Quest “Skills for Adolescence” 
drug education program: First-year behavior outcomes. 
Addictive Behaviors, 27, 619-632. 

Did not meet WWC evidence screens 
Gilman, D. A. (1991). A study of the QUEST program for the 
academic year 1990-91. A report prepared for the analysis of 
pretest and posttest measures for the North Gibson School 
Corporation. Princeton, Indiana. 6 


Goldsmith, L. M. (1990). An evaluation of the influence of the 
Skills for Adolescence program on the self-esteem and 
attitude towards school of sixth-grade Mexican-American 
students. Dissertation, Baylor University. 7 

Heinemann, G. H. (1990). The effects of the Lions-Quest “Skills 
for Adolescence” program on self-esteem development and 
academic achievement at the middle school level. Disserta- 
tion Abstracts International. (UMI No. 9033128). 8 

Heuer, L. G. (1996). Behavior, attitudes, and knowledge related 
to drug and alcohol prevention curricula in North Dakota sev- 
enth through twelve grade students. Dissertation Abstracts 
International, 56(10), 3826. 9 

Laird, M., Syropolous, M., & Black, S. (1995). An evaluation of 
Lions-Quest’s Skills for Adolescence. Limited circulation 
report. 10 

Buscemi, M. (n.d.). Lions-Quest Skills for Adolescence evalua- 
tion. Limited circulation summary report compiled by and for 
Lions-Quest. 

Additional sources: 

Keister, S. C. (n.d.). Quest International. Lions-Quest Skills for 
Adolescence. Limited circulation summary report compiled 
for and by Lions-Quest. 


5. One single-case design study was identified but is not included in this review because the WWC does not yet have standards for reviewing single-case design studies. 

6. Does not use a strong causal design: the study did not use a comparison group. 

7. Does not use a strong causal design: there was only one intervention and one comparison unit, so the analysis could not separate the effects of the intervention from other 
factors. 

8. Does not use a strong causal design: the study, which used a quasi-experimental design, did not establish that the comparison group was equivalent to the intervention group 
at the baseline. 

9. The sample is not appropriate for this review: data were not disagregated so that the WWC could not examine the results for the sample that is relevant to this review. 

10. Complete data are not reported: the WWC could not compute effect sizes. 

11. Incomparable groups: The intervention and comparison groups were not comparable at baseline and that difference was not controlled for in the statistical analysis. 
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Ray, N. G. (1990). The effects of the Lions-Quest “Skills for Roberson, R. M. (1991). The effects of the Quest’s Skills for Ado- 

Adolescence” program on student self-concept at the middle lescence training on the self concept and reading achieve- 
school level. Dissertation Abstracts International. (UMI No. ment of sixth grade students. Dissertation, US International 

9107327). 11 University. 12 


For more information about specific studies and WWC calculations, please see the WWC Skills for Adolescence 
Technical Appendices . 


12. Severe overall attrition: the study, which used a quasi-experimental design, lost a large portion of its sample from the pretest to the posttest. 
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Appendix 


Appendix A1 Study characteristics: Eisen, Zellman, & Murray, 2003 (randomized controlled trial) 


Characteristic 

Description 

Study citation 

Eisen, M., Zellman, G. L, & Murray, D. M. (2003). Evaluating the Lions-Quest “Skills for Adolescence” drug education program: Second-year behavior outcomes. Addictive 
Behaviors, 28, 883-897. 

Participants 

Participants of this study included 7,426 students who were followed from sixth through eighth grade. Female students comprised 52% of the sample. The distribution of 
minority students was as follows: 34% Hispanic, 18% African-American, and 7% Asian-American. 

Setting 

The study took place in 34 middle schools from four school districts located in three large metropolitan areas (Los Angeles-Long Beach, California; Detroit-Wayne County, 
Michigan; and Washington, DC— Baltimore, Maryland). 

Intervention 

A one-semester, 40-session Skills for Adolescence curriculum was implemented in English or Spanish in the intervention schools. Each session lasted 35-45 minutes. No 
information was provided on implementation fidelity other than that teachers were required to schedule and teach 8 of the 40 sessions that included drug prevention compo- 
nents, knowing that they may be observed by project staff and consenting to this observation. 1 

Comparison 

The comparison group “received their usual drug education programming” and were left to the discretion of the teachers at each school. A range of drug prevention programs 
and related practices were implemented in the comparison schools, including school assemblies, teacher-devised classroom curricula, and exposure to the Drug Abuse 
Resistance Education (DARE) program. The comparison schools were promised implementation of Skills for Adolescence once the final follow-up data were collected. 

Primary outcomes 
and measurement 

Outcome measures examined student self-reported cigarette smoking and alcohol, marijuana, and other illicit drug use prevalence rates. (See Appendix A2 for more detailed 
descriptions of outcome measures.) 2 

Teacher training 

The teachers attended a three-day workshop conducted by Quest International certified trainers and received teacher manuals and workbooks for each of their students. 


1 . The WWC requested information from the first study author about the timing of intervention delivery. The study author reported that in some schools the intervention was implemented during the 
first semester, while in other schools the intervention was implemented later in the academic year. 

2. Four additional student outcomes measuring attitudes were examined in this study: intentions to use tobacco, alcohol, and other drugs; perceived harm of tobacco, alcohol, and other drugs; 
refusal self-efficacy (to peer pressure); and perceived peer (close friend) use of tobacco, alcohol, and other drugs. These student outcomes were not reviewed because of lack of statistical informa- 
tion for computing effect sizes. 
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Appendix A2 Outcome measures in the behavior domain 


Outcome measure 

Description 

Alcohol use — lifetime 

One item on which students indicated if they had ever drunk alcohol. This item was adapted from the Monitoring the Future Study (MTF; as cited in Eisen, Zellman, & 
Murray, 2003). 

Alcohol use— last 30 days 

One item on which students indicated if they had drunk alcohol in the last 30 days. This item was adapted from the Monitoring the Future Study (MTF; as cited in Eisen, 
Zellman, & Murray, 2003). 

Binge drinking 

One item on which students indicated whether they had engaged in excessive drinking for three or more days during the last 30 days. This item was adapted from the 
Monitoring the Future Study (MTF; as cited in Eisen, Zellman, & Murray, 2003). 

Cigarette smoking — lifetime 

One item on which students indicated if they had ever smoked cigarettes. This outcome measure was among the items established by the National Cancer Institute as a 
standard for surveying prevalence rates (as cited in Eisen, Zellman, & Murray, 2003). 

Cigarette smoking — 
last 30 days 

One item on which students indicated if they had smoked cigarettes in the last 30 days. This outcome measure was among the items established by the National Cancer 
Institute as a standard for surveying prevalence rates (as cited in Eisen, Zellman, & Murray, 2003). 

Marijuana use — lifetime 

One item on which students indicated if they had ever used marijuana. This item was adapted from the Monitoring the Future Study (MTF; as cited in Eisen, Zellman, & 
Murray, 2003). 

Marijuana use— last 30 days 

One item on which students indicated if they had used marijuana in the last 30 days. This item was adapted from the Monitoring the Future Study (MTF; as cited in Eisen, 
Zellman, & Murray, 2003). 

Other illicit drug use — lifetime 

One item on which students indicated if they had ever used other illicit drugs. This item was adapted from the Monitoring the Future Study (MTF; as cited in Eisen, Zellman, 
& Murray, 2003). 

Other illicit drug 
use— last 30 days 

One item on which students indicated if they had used other illicit drugs in the last 30 days. This item was adapted from the Monitoring the Future Study (MTF; as cited in 
Eisen, Zellman, & Murray, 2003). 
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Appendix A3 Summary of study findings included in the rating for the behavior domain 1 





Author’s findings from the study 







Mean outcome 2 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 3 
(students/ 
schools) 

Skills for 

Adolescence Comparison 

group group 

Mean difference 4 
(Skills for 
Adolescence - 
comparison) 

Statistical 
significance 6 
Effect size 5 (at a = 0.05) 

Improvement 

index 7 


Eisen, Zellman, & Murray, 2003 (randomized controlled trial) 


Alcohol use — lifetime 

Grade 8 

5316 to 
5610/34 

33.03 

33.67 

-0.64 

-0.02 

ns 

-1 

Alcohol use — last 30 days 

Grade 8 

5316 to 
5610/34 

77.15 

76.82 

0.33 

0.01 

ns 

+0 

Cigarettes smoking — lifetime 

Grade 8 

5316 to 
5610/34 

72.00 

72.50 

-0.50 

-0.02 

ns 

-1 

Cigarettes smoking — 
last 30 days 

Grade 8 

5316 to 
5610/34 

87.53 

88.52 

-0.99 

-0.06 

ns 

-2 

Marijuana use — lifetime 

Grade 8 

5316 to 
5610/34 

72.76 

69.50 

3.26 

0.10 

ns 

+4 

Marijuana use — last 30 days 

Grade 8 

5316 to 
5610/34 

88.68 

86.21 

2.47 

0.14 

ns 

+5 

Other illicit substances 
use — lifetime 

Grade 8 

5316 to 
5610/34 

81.05 

81.56 

-0.51 

-0.02 

ns 

-1 

Other illicit substances 
use — last 30 days 

Grade 8 

5316 to 
5610/34 

93.11 

93.02 

0.09 

0.01 

ns 

+0 

Binge drinking — last 30 days 

Grade 8, baseline 
binge drinkers 

5078 to 
5359/34 

73.00 

63.00 

10.00 

0.28 

Statistically 

significant 

+11 

Binge drinking — last 30 days 

Grade 8, baseline 
nonbinge drinkers 

237 to 
251/34 

88.00 

88.00 

0.00 

0.00 

ns 

+0 

Domain average 8 for behavior 






0.04 

ns 

+2 


ns = not statistically significant 

1 . This appendix reports follow-up findings (measured toward the end of eighth grade) considered for effectiveness and the improvement index. Findings reported from the same study for the same students toward the end of seventh 
grade are not included in the ratings, but are reported in Appendix A4.1 . An additional follow-up finding (at grade 8) for binge drinking for the entire sample is presented in Appendix A4.2. 

2. The mean outcome for each study group is the percentage of participants who, based on self reporting, did not engage in a given type of problem behavior. 

3. The number of students in the analytic sample varied across outcomes, ranging from 5,316 to 5,610 students. The calculation of effect size was based on the middle point of this range (5,463). The number of baseline binge drinkers 
and nonbinge drinkers was estimated for statistical significance calculations based on the reported proportions and sample size for eighth-grade students. 
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Appendix A3 Summary of study findings included in the rating for the behavior domain 3 (continued) 


4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The study presents the percentage of students who reported involvement in each type of problem 
behavior. The percentages reported in this appendix show the proportion of students with favorable outcomes (nonusers of tobacco, alcohol, and other illicit drugs). So the signs of the mean differences presented here are the reverse of 
those reported by the study authors. 

5. Effect sizes were calculated using the odds ratio formula for binary outcomes, which were then converted to standardized mean differences (Hedges's g). For an explanation of the effect size calculation, please see the Technical Details 

of WWC-Conducted Computations . 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and, where necessary, cor- 
rects for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See the Technical Details of WWC-Conducted Computations for the 
formulas the WWC used to calculate statistical significance. In the case of Skills for Adolescence, corrections for multiple comparisons were needed. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 

8. The WWC-computed domain effect sizes for each study and for each domain across studies are a simple average rounded to two decimal places. The improvement indices are calculated from the average effect sizes. 
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Appendix A4.1 


Summary of findings of short-term effects for the behavior domain 1 





Author’s findings from the study 







Mean outcome 2 


WWC calculations 


Outcome measure 3 

Study 

sample 

Sample size 4 
(students/ 
schools) 

Skills for 

Adolescence Comparison 

group group 

Mean difference 5 
(Skills for 
Adolescence 
- comparison) 

Statistical 
significance 7 
Effect size 6 (at a = 0.05) 

Improvement 

index 8 


Eisen, Zellman, & Murray, 2003 (randomized controlled trial) 


Alcohol — lifetime 

Grade 7 

4,106 to 
5,644/34 

70.39 

69.81 

0.58 

0.02 

ns 

+1 

Alcohol — last 30 days 

Grade 7 

4,106 to 
5,644/34 

92.83 

92.75 

0.08 

0.01 

ns 

+0 

Binge drinking — last 30 days 

Grade 7 

4,106 to 
5,644/34 

96.85 

96.42 

0.43 

0.08 

ns 

+3 

Cigarettes — lifetime 

Grade 7 

4,106 to 
5,644/34 

71.78 

73.31 

-1.53 

-0.05 

ns 

-2 

Cigarettes — last 30 days 

Grade 7 

4,106 to 
5,644/34 

97.12 

96.11 

1.01 

0.19 

ns 

+7 

Marijuana — lifetime 

Grade 7 

4,106 to 
5,644/34 

90.57 

88.24 

2.33 

0.15 

ns 

+6 

Marijuana — last 30 days 

Grade 7 

4,106 to 
5,644/34 

95.72 

94.56 

1.16 

0.15 

ns 

+6 

Other illicit drugs — lifetime 

Grade 7 

4,106 to 
5,644/34 

92.97 

93.64 

-0.67 

-0.07 

ns 

-3 

Other illicit drugs — last 30 days 

Grade 7 

4,106 to 
5,644/34 

96.64 

96.45 

0.19 

0.03 

ns 

+1 

Alcohol — lifetime 

Grade 7, Hispanic 

nr/29 

67.31 

63.42 

3.89 

0.10 

ns 

+4 

Alcohol — lifetime 

Grade 7, 
non-Hispanic 

nr/29 

71.43 

74.03 

-2.6 

-0.08 

ns 

-3 

Alcohol — last 30 days 

Grade 7, Hispanic 

nr/29 

93.08 

89.64 

3.44 

0.27 

Statistically 

significant 

+11 

Alcohol — last 30 days 

Grade 7, 
non-Hispanic 

nr/29 

92.82 

94.41 

-1.59 

-0.16 

ns 

-6 


(continued) 
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Appendix A4.1 Summary of study findings included in the rating for the knowledge, attitudes, and values domain 1 (continued) 





Author’s findings from the study 
Mean outcome 2 


WWC calculations 







Mean difference 5 






Sample size 4 

Skills for 


(Skills for 


Statistical 



Study 

(students/ 

Adolescence 

Comparison 

Adolescence 


significance 7 

Improvement 

Outcome measure 3 

sample 

schools) 

group 

group 

- comparison) 

Effect size 6 

(at a = 0.05) 

index 8 


Binge drinking — last 30 days 

Grade 7, Hispanic 

nr/29 

96.79 

93.77 

3.02 

0.42 

Statistically 

significant 

+16 

Binge drinking — last 30 days 

Grade 7, 
non-Hispanic 

nr/29 

97.08 

97.69 

-0.61 

-0.15 

ns 

-6 

Alcohol — last 30 days 

Grade 7, past binge 
drinkers 

nr/29 

83.02 

79.55 

3.47 

0.14 

ns 

+6 

Alcohol — last 30 days 

Grade 7, past 
cigarette smokers 

nr/29 

91.98 

87.21 

4.77 

0.32 

Statistically 

significant 

+12 

Alcohol — last 30 days 

Grade 7, past 
marijuana users 

nr/29 

83.19 

76.48 

6.71 

0.25 

ns 

+10 


ns = not statistically significant 
nr = not reported 

1 . This appendix presents findings that fall in the behavior domain measured toward the end of seventh grade. Follow-up findings (measured toward the end of eighth grade) were used for rating purposes and are presented in Appendix A3. 

2. The mean outcome for each group is the percentage of participants who, based on self reporting, did not engage in a given type of problem behavior. 

3. The study also reported student outcomes for binge drinking by past cigarette smokers and binge drinkers and recent cigarette use by past marijuana users. These analyses were not reviewed due to severe attrition of schools from the 
analysis sample. 

4. The number of students varied by outcome, ranging from 4,106 to 5,644 students. The effect size calculation was based on the middle point of this range (4,875). 

5. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The study presents the percentage of students who reported involvement in each type of problem 
behavior. So the signs of the mean difference presented here are the reverse of those reported by the study authors. 

6. Effect size was calculated using the odds ratio formula for binary outcomes, which were then converted to standardized mean differences (Hedges's g). For an explanation of the effect size calculation, please see the Technical Details of 

WWC-Conducted Computations . 

7. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and, where necessary, 
corrects for clustering within classrooms or schools. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See the technical Details of WWC-Conducted Computations for the formulas the WWC used to 
calculate statistical significance. In the case of Skills for Adolescence, no corrections for clustering were needed. 

8. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 
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Appendix A4.2 Summary of other findings for the behavior domain 1 





Author’s findings from the study 







Mean outcome 
(standard deviation 2 ) 


WWC calculations 


Outcome measure 

Study 

sample 

Sample size 
(students/ 
schools) 

Skills for 

Adolescence Comparison 

group group 

Mean difference 3 
(Skills for 
Adolescence 
- comparison) 

Statistical 
significance 5 
Effect size 4 (at a = 0.05) 

Improvement 

index 6 




Eisen, Zellman, & Murray, 2003 (randomized controlled trial) 




Binge drinking — last 30 days 

Grade 8 

5316 to 87.33 86.89 0.44 

5610/34 

0.02 

ns 

+1 


ns = not statistically significant 

1 . This appendix presents an additional finding, binge drinking, that falls in the behavior domain. This finding is presented here for the entire sample, while findings for binge drinking behavior of past binge drinkers and nondrinkers were 
used for rating purposes and are presented in Appendix A3. 

2. The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. The study presents the percentage of students who reported involvement in each type of problem 
behavior. The percentages reported in this appendix show the proportion of students with favorable outcomes (nonusers of tobacco, alcohol, and other illicit drugs). So the signs of the mean differences presented here are the reverse of 
those reported by the study authors. 

4. For an explanation of the effect size calculation, please see the Technical Details of WWC-Conducted Computations . 

5. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between groups. The level of statistical significance was calculated by the WWC and, where necessary, 
corrects for clustering within classrooms or schools. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch . See the technical Details of WWC-Conducted Computations for the formulas the WWC used to 
calculate statistical significance. In the case of Skills for Adolescence, no corrections for clustering were needed. 

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and +50, with positive numbers denoting favorable results. 
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Appendix A5 Skills for Adolescence rating for the behavior domain 


The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1 

For the outcome domain of behavior, the WWC rated Skills for Adolescence as having potentially positive effects. It did not meet the criteria for positive effects, 
because both studies that met WWC standards with reservations did not use a strong design, according to WWC criteria. The remaining ratings (mixed effects, no 
discernible effects, potentially negative effects, and negative effects) were not considered because Skills for Adolescence was assigned the highest applicable rating. 

Rating received 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important positive effect. 

Met. The one study on Skills for Adolescence that met WWC standards showed a statistically significant positive effect on one student outcome. 

• Criterion 2: No studies showing a statistically significant or substantively important negative effect. Fewer or the same number of studies showing indeterminate 
effects than showing statistically significant or substantively important positive effects. 

Met. The WWC analysis found no indeterminate effects or statistically significant or substantively important negative effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. Skills for Adolescence had only one study showing a statistically significant positive effect. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC analysis found no statistically significant or substantively important negative effects. 


1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effect for ratings of 
potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. 
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